Troubleshoot the Content Pack for Unix Dashboards and Reports
The Content Pack for Unix Dashboards and Reports relies on the Splunk Add-on for Unix and Linux for input collection and knowledge management. When troubleshooting, determine whether the issue you are experiencing is relevant to the content pack or to the add-on.
Here are some common issues in the Content Pack for Unix Dashboards and Reports and how to resolve them.
The content pack isn't working as expected
Problem
The content pack might not work in the following ways:
- Configurations aren't reflected
- Alert Model not working properly
Cause
There is a conflict with the knowledge objects in the content pack and the Splunk App for Unix and Linux.
Solution
Check if the Splunk App for Unix and Linux is enabled on the same instance and disable it. If the Splunk App for Unix and Linux is not disabled, the Content Pack for Unix Dashboards and Reports uses the configurations and the knowledge object definitions from the app. Enabling the app results in a knowledge object conflict.
Alert Model or Open in Search features aren't working in the cloud environment
Problem
The Alert Model and Open in Search features aren't working on the Alerts - Unix dashboard.
Solution
You need to adjust alert settings.
First, add a schedule_as=classic
setting to each of the following alerts:
- Memory_Exceeds_MB_by_Process
- Memory_Exceeds_Percent_by_Host
- Memory_Exceeds_MB_by_Host
- CPU_Exceeds_Percent_by_Host
- CPU_Under_Percent_by_Host
- Load_Exceeds_by_Host
- Threads_Exceeds_by_Host
- Processes_Exceeds_by_Host
- Disk_Used_Exceeds_Percent_by_Host
- Open_Files_Exceeds_by_Process
- IO_Wait_Exceeds_Threshold
- IO_Utilization_Exceeds_Threshold
Next, follow the steps that match your deployment type:
Steps for search head cluster deployments
- Create a new file named savedsearches.conf.
- Save the file in the $SPLUNK_HOME/splunk/etc/shcluster/apps/DA-ITSI-CP-unix-dashboards/local directory on the deployer.
- Add a
schedule_as=classic
setting in each alert.[Memory_Exceeds_MB_by_Process] schedule_as=classic
- Push the updated app bundle from the deployer. The deployer restarts all the search head cluster members after the update is applied. If the deployer doesn't restart the search head cluster members, perform a rolling restart.
Steps for dedicated search head deployments
- Create a new file named savedsearches.conf.
- Save the file in the $SPLUNK_HOME/splunk/etc/apps/DA-ITSI-CP-unix-dashboards/local directory on the search head.
- Add a
schedule_as=classic
setting in each alert.[Memory_Exceeds_MB_by_Process] schedule_as=classic
- Restart the Splunk instance.
The chart bubble color differs from the actual value
Problem
The bubble in the chart shows the value of the selected parameter from the menus.
Cause
The color bar sets the color of the bubble, and the color bar shows a value between 1 to 100. If the bubble value is greater than 100, then the value is log-scaled to keep the number under 100.
The Categories tab in the Settings dashboard is stuck loading
Problem
The Categories tab in the Settings dashboard is stuck loading when opened for the first time.
Cause
The Categories tab uses the dropdown.csv file to display the default category. This file is created by the saved search at runtime after you install the content pack. If there is a high number of scheduled saved searches on the search head, the saved search for creating the dropdown.csv file is not run.
Solution
Perform the following steps to save the search manually and resolve the issue:
- Navigate to Settings > Searches, Reports, and Alerts.
- Find and run the saved search dropdowns_lookup_migrate.
CPU information isn't displaying
Problem
CPU information isn't displaying.
Cause
Software dependencies are not installed on the forwarder instance.
Solution
Ensure that all software dependencies are installed on the forwarder instance as described in the requirements for the Splunk Add-on for Unix and Linux. See Hardware and software requirements for the Splunk Add-on for Unix and Linux in the Splunk Add-on for Unix and Linux (Legacy) manual.
Split pctCPU
Problem
Unable to split the value of pctCPU between individual cores.
Cause
The value of pctCPU is designed to calculate across all CPUs, and not individual cores
Solution
You can use a search like the following example to split pctCPU into smaller units:
Search | Description |
---|---|
tag=cpu | stats avg(pctUser) | average cpu.user over all CPUs |
tag=cpu | stats avg(pctUser) by CPU | average cpu.user per CPU |
tag=cpu CPU=1 | stats avg(pctUser) by CPU | average cpu.user of CPU 1 |
Unable to change colors in the radial graph
Problem
Unable to change colors in the radial graph on the Home dashboard.
Solution
Move down the second color picker and cross it with the first color picker. Then the bottom-most color does not update.
To reflect your changes, refresh the page.
Unable to configure the Alerts and Your Data tabs
Problem
Unable to configure Alerts and Your Data tabs in the Settings dashboard.
Cause
Alerts present in the Alerts tab, and the Indexes and Sourcetypes definition in the Your Data tab, are only configurable by the Admin user.
Solution
Ensure that the current user has the admin/sc-admin role.
Could not load lookup=LOOKUP-dropdowns error
Problem
On running searches, you see a "Could not load lookup=LOOKUP-dropdowns" error in a search-head cluster environment.
Cause
The Content Pack for Unix Dashboards and Reports has a saved search which runs on startup to create the dropdowns.csv lookup. This lookup might not replicate in all the search heads and will result in this error.
Solution
Manually run the dropdowns_lookup_migrate
saved search on the search head.
Settings - Unix dashboard: Alerts aren't loading on the settings page
Problem
Alerts aren't loading on the Settings-Unix dashboard (504 gateway timeout).
Cause
Additional configs are needed to run ITSI in the Victoria Experience.
Solution
Contact Splunk Cloud Tech Ops to update your ITSI configs. To file a ticket on the Splunk Support Portal, see Support and Services.
Use the Alerts dashboard | Reports reference for the Content Pack for Unix Dashboards and Reports |
This documentation applies to the following versions of Content Pack for Unix Dashboards and Reports: 1.2.0
Feedback submitted, thanks!